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FOREWORD 

The Software Formal Inspections Guidebook is designed to support 
the inspection process of software developed by and for NASA. 

This document provides information on how to implement a 
recommended and proven method for conducting formal inspections 
of NASA software . 

This Guidebook is a companion document to NASA Standard 2202-93, 
Software Formal Inspections Standard, approved April 1993, which 
provides the rules, procedures, and specific requirements for 
conducting software formal inspections. Application of the Formal 
Inspections Standard is optional to NASA program or project 
management . In cases where program or project management decide 
to use the formal inspections method, this Guidebook provides 
additional information on how to establish and implement the 
process . 

The goal of the formal inspections process as documented in the 
above-mentioned Standard and this Guidebook is to provide a 
framework and model for an inspection process that will enable 
the detection and elimination of defects as early as possible in 
the software life cycle . An ancillary aspect of the formal 
inspection process incorporates the collection and analysis of 
inspection data to effect continual improvement in the inspection 
process and the quality of the software subjected to the process . 

The software formal inspection process supports NASA Management 
Instruction (NMI) 2410. 10B, NASA Software Management, Assurance, 
and Engineering Policy, effective April 20, 1993. 

This document has been prepared under the direction of the 
personnel shown on the following page. Specific questions 
concerning this publication should be referred to one of them, 
while general questions should be referred to the Office of 
Safety and Mission Assurance, NASA Headquarters, Washington, D.C. 
20546 . 


Original signed by 


Charles W. Mertz 

Acting Associate Administrator for Safety and Mission Assurance 



I . GENERAL 


Software formal inspections are in-process technical reviews of a 
product of the software life cycle conducted for the purpose of 
finding and eliminating defects. Formal inspections may be 
applied to any product or partial product of the software 
development process, including requirements, design, and code. 
Formal inspections are embedded in the process of developing 
products and are done in the early stages of each product's 
development . 

Formal inspections were developed by Michael Fagan at IBM to 
improve software quality and increase programmer productivity. As 
such, the formal inspection process involves the interaction of 
the following five elements: 

o Well-defined inspection steps 

o Well-defined roles for participants (moderator, recorder, 
reader author, inspector) 

o Formal collection of process and product data 

o The product being inspected 

o A supporting infrastructure 

The formal inspection process was designed to help the developing 
organization produce a better product . The process also provides 
other advantages . As defects are found and fixed the quality of 
the product increases . A more technically correct base is 
available for the each new phase of development . The overall 
software life cycle cost is lower since defects are found early 
and are easier and less expensive to fix. The effectiveness of the 
test activity is increased and less time may have to be devoted to 
testing the product . Another important benefit of formal 
inspections is the immediate evaluation and feedback to the author 
from his/her peers which will bring about improvements in the 
quality of future products. 

This guidebook describes the formal inspection process, shows 
where formal inspections fit during each life cycle phase, and 
provides some suggestions on how a formal inspection program may 
be started and improved. It is intended as a tutorial 
introduction to the inspection process. The Software Formal 
Inspection Process Standard, NASA-STD-2202-93 should be cited and 
used where formal definition of the process is needed, such as in 
process requirements and contracts. 

II. CONCEPTS AND DEFINITIONS 

A software formal inspection is a defect detection, defect 
elimination, and correction verification process carried out by a 
small group during the development life cycle . A defect is any 
error, nonconformance, or failure to satisfy a requirement in a 
product . The goal of formal inspections is to ensure that defects 



are fixed early in the life cycle rather than late, when they are 
harder to find and more costly to fix. Formal inspections are 
held throughout the software development life cycle phases to 
correct the products and to provide a short feedback loop for 
authors . 

Formal inspections are distinguished from other types of peer 
reviews in that they follow a well-defined, tried, and proven 
process for evaluating, in detail, the actual product or partial 
product with the intent of finding defects (see Section III) . 

They are conducted by individuals from development, test, and 
assurance, and may include users. Formal inspections are more 
rigorous and well-defined than other peer review processes such as 
walkthroughs, and significantly more effective. Inspections do 
not take the place of milestone reviews, status reviews, or 
testing . 

Table 1 provides a list of candidate work products for formal 
inspections. The first category, Typical Work Products, indicates 
the most likely candidates for inspections. The designations in 
parentheses are commonly used by Fagan and others and are included 
for reference. The second category in Table 1, Additional 
Candidate Work Products, shows that virtually any product that has 
requirements, constraints, and guidelines can be examined by using 
the formal inspection process . 

Inspections should be used to judge the quality of the software 
work product, not the quality of the people who create the 
product. For this reason, managers should neither participate in 
nor attend the inspection meeting. In addition, the results of 
inspections should be presented to management either statistically 
or as results for groups of products. This grouping of results 
will show managers the value of the inspection process without 
singling out any author. Using the inspection process to judge 
the capability of authors may result in less than honest and 
thorough results/ that is, co-workers may be reluctant to identify 
defects if finding them will result in a poor performance review 
for a colleague . 

Formal inspections are performed by a team that may be comprised 
of a moderator, reader, recorder, author, and other inspectors. 

The actual team size should consist of four to seven persons, 
although a minimum of three is acceptable. Each team member has a 
specific, defined role. In addition, it is the responsibility of 
the entire inspection team to find and report defects; therefore, 
all members of the team are considered inspectors . The 
moderator' s primary responsibility is to accomplish a good 
inspection. In addition, the moderator selects team members, 
prepares the team for the inspection, conducts the inspection 
meeting, and reports on the results. The reader's responsibility 
is to guide the team through the work product . The recorder' s 
responsibility is to accurately report each defect during the 
inspection meeting. The author's responsibility is to answer 
inspectors' questions and, after the inspection meeting, correct 
found defects. In addition, support may be provided to the 
inspection process by the project library, which may assist the 
moderator in keeping the status and statistics of formal 



inspections . Refer to Section IV for more extensive information 
on each role . 

Entrance and exit criteria are used to determine if a product is 
ready to be inspected (entrance) and has successfully completed 
the process (exit) . Entrance criteria to be satisfied depend on 
the product, but generally involve a determination that the 
product is mature enough to be used as is after defects are 
removed. For example, entrance criteria for a code inspection are 
usually that the code has been compiled successfully and run 
through any static analyzers used by the project, such as an 
automated standards checker. It should not, however, have been 
tested. Exit criteria are used mainly to assure that major 
defects found during the inspection process have been corrected. 
Correction of minor defects, those that will have no impact on the 
use of the product, may not be required by exit criteria. All 
work on the product to be inspected should cease once it has been 
submitted for inspection, otherwise the purpose of the inspection 
process will be defeated. Thus, the inspection must be scheduled 
in a timely manner. 

The formal inspection process and its results are supported and 
documented by a set of forms . Some of the forms that may be used 
are: Inspection Announcement, completed by the moderator, that 

notifies participants of the inspection date, time, location, and 
other important information/ Preparation Logs, completed by each 
inspector, that list defects found and time spent in preparation; 
and an Inspection Defect List, completed by the recorder, to 
provide information on each defect. In addition, the moderator 
should complete a Detailed Inspection Report at the end of each 
inspection. Other useful forms include checklists that provide 
guidance for inspectors in finding possible defects, and the 
Inspection Summary Report that summarizes data found during the 
inspection. Refer to Appendix A for sample checklists, and 
Appendix B for sample inspection forms. 

III. THE FORMAL INSPECTION PROCESS 

In order to use the formal inspection technique to its fullest 
extent, i.e., to increase product quality while maintaining cost 
effectiveness, it is important to follow the procedures that have 
been tested and refined. A formal inspection is accomplished 
through a series of steps or stages using a trained inspection 
team. Team members include the moderator, reader, recorder, 
author, and other inspectors. Figure 1 is a chart of the formal 
inspection process and team members. 

The seven stages that comprise the inspection process are 
described briefly below. In addition, if reinspection is 
required, several stages may be repeated if a high number of 
defects were found during the inspection or if defects required 
complex corrections. Determination of the need for reinspection 
is done by the inspection team at the end of the inspection 
meeting . 

Planning The period of time used to determine whether 

the product to be inspected meets the entry 
criteria, plan the inspection, select the 



team, assign roles, schedule the inspection 
meeting, and prepare and distribute the 
inspection forms materials. In addition, a 
determination is made on whether to hold an 
overview . 

Overview The optional stage in which the inspection 

team is provided with background information 
for the inspection. This stage may not be 
necessary if the team is already familiar with 
the product being inspected. 

Preparation Key stage in which members of the inspection 

team prepare individually for the inspection 
by reviewing and finding potential defects in 
the product being inspected. Potential defects 
found by individuals are discussed during the 
actual inspection meeting. 

Inspection Meeting Meeting where team members as a group review 

the product to find, categorize, and record, 
but not resolve, defects. 

Optional additional time, apart from the 
inspection meeting, that can be used for 
discussion, possible solutions, or closure of 
open issues raised in the inspection meeting. 

Stage when the author corrects defects found 
during the inspection. 

Short meeting between the moderator and author 
to determine if defects found during the 
inspection meeting have been corrected and to 
ensure that no additional defects have been 
introduced . 

Each stage of the formal inspection process is addressed more 
completely in the following sections . The responsibilities of 
each member of the inspection team are described in Section IV. 

A . Planning 

Activities in the planning stage are performed primarily by the 
moderator, who assures that the product is ready for inspection, 
selects the inspection team and assigns roles, schedules the 
meeting place and time, and assures the distribution of inspection 
materials . Inspection materials include the product to be 
inspected, the inspection announcement, the preparation log, 
background information, and checklists. 

Entrance criteria are used to screen out products that are not 
ready for inspection. For example, the entrance criteria for 
inspection of code should require that a clean compilation has 
been achieved. Entrance criteria should also specify that 
available automatic tool checking (using such tools as static 
analyzers, spell checkers, CASE tools, compilers, etc.) should be 
performed prior to distributing material to the inspection team. 


Third Hour 


Rework 


Follow-up 



The moderator determines whether the product to be inspected is of 
reasonable size so that the inspection can be completed in one 
meeting. If not, the moderator divides the product into 
manageable pieces. Sample size criteria are given in Table 2. 

The moderator then selects members for the inspection team and 
assigns them roles (See Section IV) . An inspection package 
containing the product to be inspected, checklists, references to 
relevant higher level documents, and blank preparation logs should 
be distributed to all inspection team members . Sample checklists 
and inspection forms are provided in Appendix A and Appendix B, 
respectively . 

The moderator also must decide whether the inspection team members 
are adequately familiar with the material to be inspected or 
whether an overview must be held. The team should know the 
background material from which the product was derived (e.g., 
design and requirements for source code) . The team also should 
know how the material fits into the overall system being 
developed. In most cases, team members will be adequately 
familiar from their own work on the project. If they are not, an 
overview by the author should be scheduled. 

If there is a project library, it may support the moderator during 
the planning stage by keeping records of the items to be inspected 
and those that have been completed. Once the moderator selects 
the inspection team, the library may provide each inspector with 
the inspection package. The project library can provide copies of 
reference and backup material to the inspection team. If there is 
no project library, or if the library cannot provide the needed 
support, the moderator must perform these functions. 

B . Overview 

The overview is an optional stage that is scheduled if the 
inspection team is not familiar with the material being inspected 
and its background. At the overview, the author presents the 
rationale for the product, its relationship to the rest of the 
products being developed, its function and intended use, and the 
approach used in developing it. This information is necessary to 
the inspection team so that it can perform a successful 
inspection. Interested persons aside from the inspection team may 
be invited to attend; however, all team members must attend. 

An overview is scheduled if one or more of the following 
circumstances apply: 

o The inspection team is not familiar with the product . 
o The product is new or is being inspected for the first time, 
o Inspections are new to the project, 
o Novel techniques have been used in the product . 


C . Preparation 



Preparation is the key part of the inspection process . During 
this stage, inspection team members individually prepare for their 
roles in the inspection meeting. Each inspector reviews the 
product line by line . The inspector looks at the product for 
general problems as well as those related to his/her specific area 
of expertise. Checklists should be used during this stage for 
guidance on typical types of defects to be found in the type of 
product being inspected. In addition, the product being inspected 
is checked against higher level work products, standards, and 
interface documents to assure compliance and correctness . While 
familiarizing themselves with the product, inspectors record on 
the preparation log the defects found and the time spent during 
preparation. Completed preparation logs are submitted to the 
moderator prior to the inspection meeting. Sample checklists are 
shown in Appendix A; sample inspection forms are included in 
Appendix B . 

Prior to the inspection meeting, the moderator reviews the logs 
submitted by each inspector to determine whether the team is 
adequately prepared. The moderator also checks for trouble spots 
that may need extra attention during the inspection, common 
defects that can be categorized quickly, and areas of major 
concern. If the logs indicate that the team is not adequately 
prepared, the moderator should reschedule the inspection meeting, 
as a team not fully prepared will waste time and will not be 
effective in finding defects . Preparation Log forms are returned 
to the inspectors at the inspection meeting for their use in 
pointing out defects . 

D. Inspection Meeting 

During the inspection meeting, the inspectors review the product 
as a team. Again, the focus of the meeting is finding defects. 
Briefly, the activities of the inspection meeting are: the reader 

provides a logical reading and interpretation of the product, the 
author provides clarifying information as needed, and the team 
identifies defects that are classified and recorded. 

The moderator calls the meeting to order and notes the product 
being inspected. If the team is new or contains new members, the 
moderator may begin the inspection meeting by introducing team 
members, briefly describing their roles, and restating the purpose 
of the inspection and the product. 

The reader then begins a logical and orderly interpretation of the 
product . This description should note the function of items 
(e.g., paragraphs, code blocks) and their relationship to the 
product and higher level documents. The inspection meeting is 
structured so that any inspector may interrupt the reader at any 
time when an item containing a possible defect is read. If a 
short discussion of the possible defect is needed, the reading is 
halted temporarily. The reading is resumed when the defect is 
noted and categorized by the recorder on the inspection defect 
list (see Appendix B) . The moderator should try to limit 
discussions that appear to be consuming too much of the inspection 
meeting time. Imposing a time limit on discussions may be useful. 
If discussions are not completed within the time limit, the 
moderator will declare the issue unresolved and proceed with the 



meeting. The recorder should note unresolved issues as open items 
to be resolved during the third hour (see section E.) . 


The team should reach a consensus on whether each possible defect 
raised is an actual defect. Sometimes, what seems to be a 
potential defect may be a mistake on the part of the inspector or 
is only a misunderstanding that may be clarified by the author . 

If consensus cannot be reached, the potential defect should be 
recorded as an open issue to be dispositioned during the third 
hour. This ensures the right of every inspector to have each 
possible defect recorded and resolved. The recorder will itemize 
each agreed upon defect by recording its location, a brief 
description, its classification, and the inspector who found it. 

The question of whether a potential defect is a real defect may be 
resolved by reference to parent documents, which should be 
available to the inspectors. If the discussion identifies the 
parent document as potentially in error, an open issue should be 
noted and the inspection should continue. Resolution of the issue 
(whether the defect is in the product being inspected or in the 
parent document) can be done during the third hour. Open issues 
are logged on the Inspection Report form. 

To determine the priority for fixing the defects in the product, 
the inspection team or moderator should classify them by severity 
(major or minor) . For example, a defect that would cause the 
system to fail to satisfy a requirement would be classified as 
major/ all others would be classified as minor (e.g., 
typographical errors, minor standards violation, ) . Additional 
data collected from each inspection is the classification of 
defects by type such as data error, requirements compliance, 
standards compliance, logic, interface, data, performance, and 
readability. If using the Inspection Defect List sample form in 
Appendix B, this information goes in the "Type" field for each 
defect . 

At the end of the meeting, the number of defects are summarized, 
and the moderator and/or the team determines whether reinspection 
will be needed. At this time, the moderator also determines 
whether a third-hour activity is needed. If a third hour is 
needed, action items are assigned to individual inspectors at this 
time . 

The team must focus on finding defects and should not be 
concerned with other activities such as problem solving. It is 
the responsibility of the moderator to control and focus the 
meeting. To avoid fatigue and thus missing defects, the 
inspection meeting should be limited to 2 hours. If the 
inspection is not completed in the original meeting, a second 
meeting must be scheduled. 

After the meeting, the author and moderator meet briefly to 
estimate rework time and schedule the follow-up meeting. The 
author is given a copy of the defect list for reference during 
rework. Note that formal discrepancy reports (DRs) and change 
requests (CRs) are not written against defects found in the 
document or code being inspected; however, discrepancy reports 



should be written against any defects found in higher level 
documents . The reason that DRs and CRs are not required is that 
inspections take place before the product is under configuration 
control and that closure is assured by the defect list, the follow 
up stage, and by reinspections. 

E . Third Hour 

The third hour is time that is used for discussion or for closing 
open issues . A third hour should be scheduled when the author 
wishes to discuss corrections of defects found, or when open 
issues, such as a potential major defects in parent (higher level 
reference) documents, need to be dispositioned . The third hour 
may take the form of an additional meeting or of time for team 
members to perform and report on action items . It does not have 
to take place immediately after the inspection meeting and it does 
not have to be limited to 1 hour. 

When used as meeting time, the third hour provides an opportunity 
to discuss solutions and resolve disagreements. Attendees may 
include any subset or superset of the inspection team including 
relevant managers (present for technical reasons only) , outside 
technical experts, and other people who could be affected by the 
way the defect is fixed or the issue resolved. In many cases, 
only those inspectors who have a specific interest need attend. 
During a third hour meeting, the author is provided with 
information that would make rework more effective, major defects 
found in parent documents are reported, and any open issues 
remaining from the inspection are resolved. 

When used as an opportunity for individual inspectors to perform 
action items, the third hour usually is spent researching and 
dispositioning open issues, finding information to resolve a 
discrepancy, or writing discrepancy reports or change requests for 
major defects found in parent documents under configuration 
management . 

F . Rework 

The purpose of rework is to correct defects found during the 
inspection. The author is responsible for correcting all major 
defects noted in the inspection defect list. Minor defects should 
be resolved if cost and schedule permit . The moderator should 
make sure that information generated from any open issues or 
action items are communicated to and addressed by the author . 

G . Reinspection 

Reinspection may be required when there are a large number of 
defects in the product or when one or more defects require 
extensive or complicated corrections . Reinspection allows the 
changes to the product to be reviewed by the entire team instead 
of just the moderator. The moderator and the team decide the 
necessity for reinspection at the end of the inspection meeting. 


H . Follow-Up 



Follow-up is a short meeting between the moderator and the author 
to ensure that all major defects found during the inspection have 
been corrected and no secondary defects have been introduced. The 
author reviews the measures taken to correct each major defect 
with the moderator. The moderator ensures that all open issues 
have been resolved and that changes due to the resolution, if any, 
have been incorporated in the product . Corrections to minor 
defects also may be reviewed, but with less emphasis . The 
moderator ensures that the exit criteria for the type of 
inspection have been met. The moderator may include additional 
reviewers in the follow-up meeting if extra technical expertise is 
required . 

If all of the major defects have been corrected, all open issues 
have been resolved, and the product has satisfied the exit 
criteria, then the moderator "Passes" the product by recording the 
completion of inspection on the Inspection Summary Report . If 
these conditions are not met, the author returns to the rework 
stage to make the necessary changes. 

IV. THE INSPECTION TEAM 

The inspection team is a small group of peer staff members with a 
vested interest in the product . The minimum team size is three 
persons, although a typical team varies from four to seven. 

Larger teams are generally used for high level documents, while 
smaller teams are used at detailed technical levels . Members are 
added when their point of view is needed. A good mix of 
inspectors representing various areas of expertise is important to 
the inspection process . The knowledge and experience of such a 
group, each looking at the product from his/her own perspective, 
helps bring many subtle defects to light. "Synergy," where an idea 
from one team member often leads to another idea from a different 
team member, is one indication of a healthy inspection process. 

The role of each individual is explained in the following 
sections . 

A . Inspectors 

Every member of the inspection team is considered an inspector in 
addition to any other assigned role. Inspectors are responsible 
for finding defects in the work during preparation and during the 
inspection meeting. In addition to functioning as an inspector, 
some members of the inspection team will carry out roles as 
moderator, author, reader, and recorder, as appropriate. 

Primary candidates for inspectors are personnel involved with the 
product in immediately preceding, current, and following life 
cycle phases. For example, in a design inspection, good 
inspectors may be selected from the individuals who wrote the 
requirements, people who will code the design, and designers of 
interfacing segments of the system. An exception to this rule is 
that the author of a unit of code should not serve as an 
inspector for test procedures that are to be used to test that 
code, because the code author may want changes made to the test 
procedures so they will work with the code as it exists . Any 
group with an interest in the product should be considered for 
potential team members including Systems Engineering, Testing, 



Software Assurance, Systems Administrators and Operators, and 
Users . Sources for inspectors are not limited to the staff of the 
software development organization. Outside inspectors should be 
brought in when they have a particular expertise that would add to 
the effectiveness of the inspection. 

B . Moderator 

The moderator leads the inspection team and is responsible for 
ensuring that a good inspection is achieved. Because this role is 
critical to the formal inspection process, training for moderators 
is more important and extensive than that of other inspectors . 

The moderator is directly active in all stages of the inspection 
process except rework. Since acting as a moderator is time 
consuming and requires specific skills, moderators often are 
selected and trained by the development organization and then 
assigned to a specific development project. Primary duties of the 
moderator include coordinating the selection of the inspection 
team, assigning team roles, and leading the team throughout the 
process. A major function of the moderator is to ensure that the 
team keeps its emotions in check and that the inspection meeting 
is not used to find faults with the author. The moderator is also 
responsible for assuring inspection data are collected on the 
inspection report forms . 

C . Reader 

The reader is responsible for guiding the inspection team through 
the product during the inspection meeting by reading or 
paraphrasing the product . An individual who will use the product 
during the next life cycle phase is an excellent candidate for 
reader as the process of reading and paraphrasing it will cause 
this potential user to become very familiar with the product 
before it is delivered. The reader also performs the duties of a 
regular inspector. 

D . Recorder 

The recorder is responsible for accurately recording information 
during the meeting about each defect found on the inspection 
defect list. The list should include the location of the defect, 
a brief description of it, its classification, and the identity of 
the inspector who found it . The recorder also must record 
information on any issues raised and not resolved, and any defects 
that are found in parent documents . The recorder also performs 
the duties of a regular inspector. 

E . Author 

The author is the originator of the product being inspected. As 
such, she/he is primarily responsible for providing information on 
the product being inspected and for answering inspectors' 
questions to ensure that simple misunderstandings are not 
classified as defects . In addition, the author should also 
perform the duties of a regular inspector. The author is required 
to correct all major defects found during the formal inspection, 
and minor defects as time and resources allow. 



V. FORMAL INSPECTIONS DURING THE SOFTWARE LIFE CYCLE 


Formal inspections are in-process peer reviews conducted within 
the phase of the life cycle in which the product is developed. 

The following describes the life cycle phases of software 
development and suggests products that may be inspected during 
each phase . The software life cycle used is the NASA standard 
waterfall model; the material may be adapted to other life cycles 
if needed. 

A. Software Concept and Initiation Phase 

During this phase, the software concept is developed, the 
feasibility of the software system is evaluated, and the 
acquisition strategy is developed. Much of the documentation for 
the project is started or a draft provided within this phase. The 
most important document to inspect is the portion of the system 
requirements document that applies to software. This inspection 
has the shorthand designation of RO (see table 1) . Other 
candidates for inspection include system specifications and plans 
such as the software management and assurance plans . Potential 
inspectors are the users and assurers of this documentation and 
the system to be developed. 

B. Software Requirements Phase 

During this phase, the software concept and allocated system 
requirements are analyzed and documented as detailed software 
requirements . Test planning is started, with a preliminary method 
for verifying each requirement identified and included in a first 
version of a test plan. Risks are identified and planned for, and 
the size and scope of the remainder of the project is re- 
estimated. Methods, standards, and procedures are detailed and 
put in place . 

The requirements document is the product that is most typically 
inspected in this phase. This is known as the R1 inspection. The 
requirements document should be inspected for completeness and 
accuracy, for traceability to higher level documents, and to 
assure that a sufficient base is provided for the software design. 
Other documents that are produced during this life cycle phase, 
such as the draft acceptance test plan, can also be inspected 
(Inspection IT1) . Candidates for inspectors include the assurers 
and potential users of the documents, including designers, coders, 
and testers . 

C. Software Architectural (Preliminary) Design Phase 

During this phase, the overall design for the software is 
developed, allocating all of the requirements to software 
components. The architectural design inspection is designated 10. 

The design should be inspected for satisfaction of and 
traceability to the requirements, correctness, clarity, 
codability, testability, and consistency. 



D . Software Detailed Design Phase 


During this phase, the architectural design is expanded to the 
unit level . Interface control documents are completed and test 
plans updated. Constraints and system resource limits are re- 
estimated and analyzed, and staffing and test resources are 
validated . 

The detailed design should follow exactly the base-lined higher 
level design, and should be inspected for the same 
characteristics. As a secondary condition, the design should be 
inspected for satisfaction of software quality engineering 
standards such as information hiding, use of simple structures, 
etc. The detailed design inspection is designated II. 

Candidates for the inspection team may be selected from 
participants in the design, code, and test phases. 

E . Software Implementation Phase 

During this phase, the software is coded and unit tested. All 
documentation is produced in quasi-final form, including internal 
code documentation. 

Code and all new documentation are the candidates for inspections 
during this phase. Code inspections (designated 12) should check 
for technical accuracy and completeness of the code, verify that 
it implements the planned design, and ensure good coding practices 
and standards are used. Code inspections should be done after the 
code has been compiled and all syntax errors removed, but before 
it has been unit tested. Other candidates are the integration and 
test plan and procedures, and other documents that have been 
produced. Documents should be inspected for accuracy, 
completeness, and traceability to higher level documents. The 
inspection team may be selected from participants in the detailed 
design, code, test, verification and validation, or from software 
quality assurance. 

F . Software Integration and Test Phase 

During this phase the software units are integrated into a 
completed system; nonconformances are detected, documented, and 
corrected; and it is demonstrated that the system meets its 
requirements . The integration and test plan is executed, the 
software documentation is updated and completed, and the products 
are finalized for delivery. 

The final version of the Acceptance Test Plan should be inspected 
to detect defects in the definition of test cases and to verify 
that each test case will verify the requirements with which it is 
associated. This is the IT1 inspection. Test case and test 
procedure inspections should verify that they are in accord with 
one another and with the Acceptance Test Plan. These inspections 
should verify that the test cases and procedures will execute 
properly and correctly, and that all needed data are available. 
Potential inspectors are representatives from any of the life 
cycle phases before or after this one. 



While the products listed above are used in the Integration and 
Test Phase, they may have been developed in prior phases. 
Inspections should be integrated into the development process, and 
these products inspected when they are developed. If so, few or 
no inspections may actually be done during this phase; inspections 
are needed only if new test cases and procedures are developed. 

G. Software Acceptance and Delivery Phase 

The formal acceptance procedure is carried out during the 
acceptance and delivery phase. As a minimum, there is a 
requirements-driven demonstration of the software to show that it 
meets its requirements. The phase also may include acquirer 
tests, field usage, or other arrangements that are intended to 
assure that the software will function correctly in its intended 
environment . 

There is little or no inspection activity during this phase. 

H. Software Sustaining Engineering and Operations Phase 

During this phase, the software is used to achieve the objectives 
for which it was developed and acquired. Corrections and 
modifications are made to the software to sustain its operational 
capabilities and to upgrade its capacity to support its users . 
Software changes may range in scope from simple corrective action 
up to major modifications that require a full life cycle process. 

Formal inspections should be scheduled in response to the degree 
of new development activity involved. Significant new material to 
be incorporated into any product should be inspected. A useful 
technique is to have the need for inspections evaluated as part of 
the change control and configuration management process . 

VI. STARTING A FORMAL INSPECTIONS PROGRAM 

Formal inspections have been proven to be effective in detecting 
and removing defects, and to be cost effective when compared to 
the cost of finding defects by testing. However, they represent 
an up-front cost and a diversion from more traditional uses of 
software development resources. Since there may well be 
skepticism about inspections, beginning a program may find some 
resistance. Educating management in the advantages of formal 
inspections, particularly stressing how the up-front costs are 
likely to be made up by reduced testing costs may help to overcome 
the skepticism. 

A critical first step in initiating an inspection program is to 
select the class of material to first be inspected. It is 
advisable to begin with material in which everyone will agree that 
defects have to be found and removed. Based on their own 
experiences in starting inspection programs, both JPL and LaRC 
recommend starting with requirements inspections, as the benefits 
of formal inspections are shown early in the software life cycle. 


Defects in requirements also have more impact than those in other 
products. For example, for each defect found and corrected in the 



requirements, many "defects" would have been present in the design 
and code. If these resultant defects were not found until 
testing, they would cause a great deal of rework to many products . 
Such early results will be popular with management, and should 
raise enthusiasm for starting the program. Defects in 
requirements, especially, and in design, are more expensive to 
correct after the system has been implemented than are code 
defects . 

NASA experience has shown that inspection of requirements and 
design will significantly reduce code "errors; " some projects 
conduct formal inspections of all of their requirements and 
design, but only inspect critical code. Although code inspections 
may be the easiest type of inspections to perform, they may not be 
the most productive. 

Once a starting point for inspections has been selected, needed 
resources must be budgeted and roles and responsibilities decided 
upon. Resources will be needed for start-up costs, overhead 
costs, and operational costs. Start-up costs include training of 
moderators and other inspectors, development of forms and report 
formats, and acquisition of data processing resources for the 
recording and trending of inspection data. Overhead costs 
associated with the formal inspection program consist of moderator 
time for arranging and scheduling inspections and follow-up, and 
the moderator's or project librarian's role in making copies of 
materials available and keeping track of the status of items as 
they progress through the inspection process. In addition, while 
the collection and trending of data is important, it will consume 
some resources . Operational costs consist of the time spent by 
project members preparing for and participating in inspection 
meetings . 

The chief moderator is key to the whole formal inspection program 
and should be selected as part of the start-up. The chief 
moderator oversees and directs the inspection program; analyzes 
the effectiveness of the inspection process; and coordinates the 
evolution of the program, forms, and checklists. The chief 
moderator should be the moderator for the inspections on the 
initial project. When sufficient inspectors have been trained and 
become experienced, additional moderators may be selected from 
this pool and trained for new projects to which formal 
inspections are to be applied. 

If it is possible to arrange, project libraries can make both the 
start-up and the inspection process run more smoothly; this 
support will allow the moderator to concentrate on the success of 
the inspection program. The project library helps the moderator 
to maintain lists of what is to be inspected and assists with some 
of the mechanics of the inspection process such as delivering 
materials, setting up meeting rooms, etc. The library should 
provide reference documentation for the inspectors . 

Checklists to guide the inspectors may be developed from the 
samples provided in Appendix A, obtained from JPL, or developed 
specifically according to the needs of the program. In the long 
run, checklists will be needed for each type of material to be 



inspected and each major language used. For example, there should 
be checklists for requirements, design, Ada code, C code, user's 
guides, etc. At the start-up, only checklists for the limited 
items to be inspected are needed. 

Forms to record results and collect inspection data should be 
defined. Example forms are shown in Appendix B. They should be 
tailored as needed to reflect working conditions and to capture 
the specific data desired. The standard forms will evolve over 
time . 

Once the centralized resources are in place (e.g., chief 
moderator, librarian, forms, checklists, and data processing 
resources), project individuals who are to participate in the 
inspection process must be identified. Project technical people 
are the key to the program since they are the readers, inspectors, 
and the ones who have to use the inspection results to improve the 
product . Once participants have been identified, they should be 
trained. JPL has developed a formal inspection training class for 
NASA that is workshop-oriented and very effective. Alternatively, 
outside organizations have inspection training available. 

Projects introducing inspections must plan to accommodate them. 

In addition to identifying inspectors for training, managers 
should plan time in schedules for inspections, analysis, and 
rework. Experience shows that the resources used for inspections 
are more than made up for in shortened test time and the costs of 
finding and repairing defects that are imbedded in the system, but 
resources are needed at different points in the life cycle. 

Project or other management must also provide sufficient and 
appropriate space in which the inspection process can take place. 

Once resources are available and the moderator and an adequate 
number of inspectors have been trained, the inspection program can 
begin. One important factor to have in operation from the 
beginning is a data collection program. Formal inspection data is 
easy to collect because the process is very structured. The data 
can also be used to improve the processes that produce the 
products that are inspected. The subject of data collection and 
evaluation and improvement of the inspection process is discussed 
in Section VII. 

Once the inspection program is underway and there are several 
moderators, moderators should meet regularly to discuss problems 
and successes with the inspection program and suggest ways to 
improve it . This meeting should be chaired by the chief moderator 
who is responsible for carrying out or recommending improvements 
and evaluating whether the level of training and experience is 
being maintained. 

VII. PROCESS EVALUATION AND IMPROVEMENT 

Formal inspections have been demonstrated by many organizations 
to be an effective method for finding and removing defects in 
software products. However, just putting a formal inspections 
program in place does not guarantee that the program will operate 
at maximum efficiency. It is important to evaluate the 
implementation of the formal inspections process and to improve 



it by fine tuning the procedures that are followed. The items 
that most need to be tailored are the inspection rates and the 
checklists . If too large an amount of a software product is to 
be inspected at a meeting, the meeting will have to rush along 
too rapidly to be effective, or meetings will routinely have to 
be continued with resultant inefficiencies and schedule delays . 

If too little of the product is allocated to one inspection, the 
program will also not work at peak efficiency. The inspection 
rates must be tailored to the complexity of the product and the 
ability of the inspectors . Checklists should be tailored to 
ensure that inspectors pay attention to the types of errors that 
actually occur in the products being inspected. Fine tuning of 
the checklists will make more efficient use of preparation time 
and meeting time, and should help ensure that more of the defects 
are found. 

In order to evaluate the effectiveness of the inspection program, 
the data collected from inspections should be routinely analyzed 
in order to reveal trends . The trends that should be evaluated 
are the amount of product inspected at a meeting, the time taken 
in preparation and in the inspection meeting, total defects found 
per inspection, the types of defects found, and the phases in the 
development life cycle where defects were found. The data for 
trending is normally collected by the moderator, using forms 
provided for this purpose (see appendix B) . 

In the case where a trend points to a decline in efficiency in, 
for example, the time spent preparing for and conducting 
inspections, action can be taken to analyze the procedures and 
correct the problem. The analysis might lead to changes in the 
amount of product scheduled for each meeting, or to the 
checklists provided to the participants, or to the training for 
the participants . If the trend data shows fewer defects are 
being found in inspections late in the life cycle, such as code 
inspections, an analysis might show that the inspectors were not 
adequately preparing, or it might show that the organization has 
becomes so effective in performing requirements and design 
inspections that there are few code defects to be found. In this 
case, steps may be taken to modify procedures to inspect only 
critical code. 

The data from inspections may also be used in another manner. 

If, for example, the inspection data show that during code 
inspections a high percentage of code defects are due to defects 
introduced during the design phase of the life cycle, then steps 
could be taken to attempt to improve the effectiveness of both 
the design process and the design inspections. This ability to 
point out the life cycle step in which defects were introduced is 
dependent on careful data gathering, but could pay high 
dividends . Any attempt to change the life cycle processes used 
in an organization should be done with great care, and 
information from other sources than just inspections should be 
used, but the inspection data could be of great assistance. 

The evaluation process should be continuous, that is, the trend 
data should be kept up to date, it should be examined regularly, 
and the trends should be available to all participants in the 



inspections program. Only continuous monitoring can ensure the 
maximum cost effectiveness in the resources used for inspections . 


The data gathered could be used to modify the inspection process 
itself. The third hour is one such modification, introduced by 
JPL to the process defined by Michael Fagan based on their 
analysis of their inspection program. Modification of the 
inspection process should be done only after very careful 
analysis and testing of the proposed changes. The formal 
inspection process is effective because it is well defined, well 
tested, and done in exactly the same manner time after time. 

VIII. SUMMARY 


The following summarizes the essentials of the formal inspection 

process and provides a quick reference: 

1 . Inspections are carried out on products that have been 
completed by their author but not yet tested, reviewed, or 
otherwise approved or baselined. 

2 . The objective of the inspection process is to detect and 
remove defects. Typical defects are errors of documentation, 
logic, and function. 

3. Inspections are carried out by peers of the author. 
Participants in the inspection process should represent 
organizations that will use or will be affected by the 
material being inspected. 

4 . Inspections should not be used as a tool to evaluate workers . 
Management is not to be present during inspections . When a 
management official has technical expertise which is not 
available from other sources, that individual may be brought 
into the third hour. 

5. A trained moderator leads inspections, and all participants 
should have training in the process . 

6. Inspectors are assigned to and prepare for specific roles 
(e.g., reader, recorder, author ) . 

7 . Inspections are carried out in a prescribed series of steps 
from planning through follow-up. 

8. Inspection meetings are limited to two hours. 

9. Checklists of questions and typical defects are used to 
stimulate defect finding. Project-tailored entrance and exit 
criteria should be developed for each type of product to be 
inspected . 

10. The product being inspected should be of an appropriate size 
that it can be inspected during a two hour meeting. 

11. Correction of defects is the responsibility of the author, 



and is verified by the moderator. The inspection team must 
refrain from suggesting methods for correction during the 
inspection meeting. 

12. Data and trends on the number of defects, the types of 
defects, and the time expended on inspections should be 
maintained. This information should be used to evaluate and 
improve the effectiveness of the inspection program. 



